22 research outputs found
Multimodal Subspace Support Vector Data Description
In this paper, we propose a novel method for projecting data from multiple
modalities to a new subspace optimized for one-class classification. The
proposed method iteratively transforms the data from the original feature space
of each modality to a new common feature space along with finding a joint
compact description of data coming from all the modalities. For data in each
modality, we define a separate transformation to map the data from the
corresponding feature space to the new optimized subspace by exploiting the
available information from the class of interest only. We also propose
different regularization strategies for the proposed method and provide both
linear and non-linear formulations. The proposed Multimodal Subspace Support
Vector Data Description outperforms all the competing methods using data from a
single modality or fusing data from all modalities in four out of five
datasets.Comment: 26 pages manuscript (6 tables, 2 figures), 24 pages supplementary
material (27 tables, 10 figures). The manuscript and supplementary material
are combined as a single .pdf (50 pages) fil
Boosting rare benthic macroinvertebrates taxa identification with one-class classification
Insect monitoring is crucial for understanding the consequences of rapid
ecological changes, but taxa identification currently requires tedious manual
expert work and cannot be scaled-up efficiently. Deep convolutional neural
networks (CNNs), provide a viable way to significantly increase the
biomonitoring volumes. However, taxa abundances are typically very imbalanced
and the amounts of training images for the rarest classes are simply too low
for deep CNNs. As a result, the samples from the rare classes are often
completely missed, while detecting them has biological importance. In this
paper, we propose combining the trained deep CNN with one-class classifiers to
improve the rare species identification. One-class classification models are
traditionally trained with much fewer samples and they can provide a mechanism
to indicate samples potentially belonging to the rare classes for human
inspection. Our experiments confirm that the proposed approach may indeed
support moving towards partial automation of the taxa identification task.Comment: 5 pages, 1 figure, 2 table
Subspace Support Vector Data Description and Extensions
Machine learning deals with discovering the knowledge that governs the learning process. The science of machine learning helps create techniques that enhance the capabilities of a system through the use of data. Typical machine learning techniques identify or predict different patterns in the data. In classification tasks, a machine learning model is trained using some training data to identify the unknown function that maps the input data to the output labels. The classification task gets challenging if the data from some categories are either unavailable or so diverse that they cannot be modelled statistically. For example, to train a model for anomaly detection, it is usually challenging to collect anomalous data for training, but the normal data is available in abundance. In such cases, it is possible to use One-Class Classification (OCC) techniques where the model is trained by using data only from one class.
OCC algorithms are practical in situations where it is vital to identify one of the categories, but the examples from that specific category are scarce. Numerous OCC techniques have been proposed in the literature that model the data in the given feature space; however, such data can be high-dimensional or may not provide discriminative information for classification. In order to avoid the curse of dimensionality, standard dimensionality reduction techniques are commonly used as a preprocessing step in many machine learning algorithms. Principal Component Analysis (PCA) is an example of a widely used algorithm to transform data into a subspace suitable for the task at hand while maintaining the meaningful features of a given dataset.
This thesis provides a new paradigm that jointly optimizes a subspace and data description for one-class classification via Support Vector Data Description (SVDD). We initiated the idea of subspace learning for one class classification by proposing a novel Subspace Support Vector Data Description (SSVDD) method, which was further extended to Ellipsoidal Subspace Support Vector Data Description (ESSVDD). ESSVDD generalizes SSVDD for a hypersphere by using ellipsoidal data description and it converges faster than SSVDD. It is important to train a joint model for multimodal data when data is collected from multiple sources. Therefore, we also proposed a multimodal approach, namely Multimodal Subspace Support Vector Data Description (MSSVDD) for transforming the data from multiple modalities to a common shared space for OCC. An important contribution of this thesis is to provide a framework unifying the subspace learning methods for SVDD. The proposed Graph-Embedded Subspace Support Vector Data Description (GESSVDD) framework helps revealing novel insights into the previously proposed methods and allows deriving novel variants that incorporate different optimization goals.
The main focus of the thesis is on generic novel methods which can be adapted to different application domains. We experimented with standard datasets from different domains such as robotics, healthcare, and economics and achieved better performance than competing methods in most of the cases. We also proposed a taxa identification framework for rare benthic macroinvertebrates. Benthic macroinvertebrate taxa distribution is typically very imbalanced. The amounts of training images for the rarest classes are too low for properly training deep learning-based methods, while these rarest classes can be central in biodiversity monitoring. We show that the classic one-class classifiers in general, and the proposed methods in particular, can enhance a deep neural network classification performance for imbalanced datasets
Railway vehicle detection from audio recordings using one-class classification
In this thesis, we focus on detecting a train from the sound generated by it. An audio sensor is placed close to a railway track to record ambient sounds which may or may not originate from a train. In this problem, we de ne the target event as the recording of a train sound and non-target events are all other audio events that are recorded by the audio sensor. In machine learning and pattern recognition, classifiers are trained from labeled data to categorize a new observation. Classifiers are usually trained from data which contain all possible classes, however it is possible that during training the classifier, for some classes the data is either not available or it is so diverse in nature that it cannot be used reliably. In case of binary classification, if one of the classes do not have reliable training data, we can use a \one class classification" strategy which only uses single class data for training. For train detection from audio, we compared a one-class classi er called support vector data description (SVDD) with binary classifiers and showed that SVDD performs well in cases where data from the outlier class is scarce. We also tested the SVDD trained model in real time and the results indicate that the goal of reducing the false positive rate is satisfactorily achieved. The tests are performed using audio data recorded in Bathmen, a town in eastern Netherlands, by the company Sensornet for a project about railway vehicle detection and sound level monitoring
Newton Method-based Subspace Support Vector Data Description
In this paper, we present an adaptation of Newton's method for the
optimization of Subspace Support Vector Data Description (S-SVDD). The
objective of S-SVDD is to map the original data to a subspace optimized for
one-class classification, and the iterative optimization process of data
mapping and description in S-SVDD relies on gradient descent. However, gradient
descent only utilizes first-order information, which may lead to suboptimal
results. To address this limitation, we leverage Newton's method to enhance
data mapping and data description for an improved optimization of subspace
learning-based one-class classification. By incorporating this auxiliary
information, Newton's method offers a more efficient strategy for subspace
learning in one-class classification as compared to gradient-based
optimization. The paper discusses the limitations of gradient descent and the
advantages of using Newton's method in subspace learning for one-class
classification tasks. We provide both linear and nonlinear formulations of
Newton's method-based optimization for S-SVDD. In our experiments, we explored
both the minimization and maximization strategies of the objective. The results
demonstrate that the proposed optimization strategy outperforms the
gradient-based S-SVDD in most cases.Comment: 8 pages, 2 figures, 2 tables, 1 Algorithm. Accepted at IEEE Symposium
Series on Computational Intelligence 202
Subspace Support Vector Data Description
This paper proposes a novel method for solving one-class classification
problems. The proposed approach, namely Subspace Support Vector Data
Description, maps the data to a subspace that is optimized for one-class
classification. In that feature space, the optimal hypersphere enclosing the
target class is then determined. The method iteratively optimizes the data
mapping along with data description in order to define a compact class
representation in a low-dimensional feature space. We provide both linear and
non-linear mappings for the proposed method. Experiments on 14 publicly
available datasets indicate that the proposed Subspace Support Vector Data
Description provides better performance compared to baselines and other
recently proposed one-class classification methods.Comment: 6 pages, submitted/accepted, ICPR 201
Credit Card Fraud Detection with Subspace Learning-based One-Class Classification
In an increasingly digitalized commerce landscape, the proliferation of
credit card fraud and the evolution of sophisticated fraudulent techniques have
led to substantial financial losses. Automating credit card fraud detection is
a viable way to accelerate detection, reducing response times and minimizing
potential financial losses. However, addressing this challenge is complicated
by the highly imbalanced nature of the datasets, where genuine transactions
vastly outnumber fraudulent ones. Furthermore, the high number of dimensions
within the feature set gives rise to the ``curse of dimensionality". In this
paper, we investigate subspace learning-based approaches centered on One-Class
Classification (OCC) algorithms, which excel in handling imbalanced data
distributions and possess the capability to anticipate and counter the
transactions carried out by yet-to-be-invented fraud techniques. The study
highlights the potential of subspace learning-based OCC algorithms by
investigating the limitations of current fraud detection strategies and the
specific challenges of credit card fraud detection. These algorithms integrate
subspace learning into the data description; hence, the models transform the
data into a lower-dimensional subspace optimized for OCC. Through rigorous
experimentation and analysis, the study validated that the proposed approach
helps tackle the curse of dimensionality and the imbalanced nature of credit
card data for automatic fraud detection to mitigate financial losses caused by
fraudulent activities.Comment: 6 pages, 1 figure, 2 tables. Accepted at IEEE Symposium Series on
Computational Intelligence 202
Hyperspectral Image Analysis with Subspace Learning-based One-Class Classification
Hyperspectral image (HSI) classification is an important task in many
applications, such as environmental monitoring, medical imaging, and land
use/land cover (LULC) classification. Due to the significant amount of spectral
information from recent HSI sensors, analyzing the acquired images is
challenging using traditional Machine Learning (ML) methods. As the number of
frequency bands increases, the required number of training samples increases
exponentially to achieve a reasonable classification accuracy, also known as
the curse of dimensionality. Therefore, separate band selection or
dimensionality reduction techniques are often applied before performing any
classification task over HSI data. In this study, we investigate recently
proposed subspace learning methods for one-class classification (OCC). These
methods map high-dimensional data to a lower-dimensional feature space that is
optimized for one-class classification. In this way, there is no separate
dimensionality reduction or feature selection procedure needed in the proposed
classification framework. Moreover, one-class classifiers have the ability to
learn a data description from the category of a single class only. Considering
the imbalanced labels of the LULC classification problem and rich spectral
information (high number of dimensions), the proposed classification approach
is well-suited for HSI data. Overall, this is a pioneer study focusing on
subspace learning-based one-class classification for HSI data. We analyze the
performance of the proposed subspace learning one-class classifiers in the
proposed pipeline. Our experiments validate that the proposed approach helps
tackle the curse of dimensionality along with the imbalanced nature of HSI
data
One-Class Classification for Intrusion Detection on Vehicular Networks
Controller Area Network bus systems within vehicular networks are not
equipped with the tools necessary to ward off and protect themselves from
modern cyber-security threats. Work has been done on using machine learning
methods to detect and report these attacks, but common methods are not robust
towards unknown attacks. These methods usually rely on there being a sufficient
representation of attack data, which may not be available due to there either
not being enough data present to adequately represent its distribution or the
distribution itself is too diverse in nature for there to be a sufficient
representation of it. With the use of one-class classification methods, this
issue can be mitigated as only normal data is required to train a model for the
detection of anomalous instances. Research has been done on the efficacy of
these methods, most notably One-Class Support Vector Machine and Support Vector
Data Description, but many new extensions of these works have been proposed and
have yet to be tested for injection attacks in vehicular networks. In this
paper, we investigate the performance of various state-of-the-art one-class
classification methods for detecting injection attacks on Controller Area
Network bus traffic. We investigate the effectiveness of these techniques on
attacks launched on Controller Area Network buses from two different vehicles
during normal operation and while being attacked. We observe that the Subspace
Support Vector Data Description method outperformed all other tested methods
with a Gmean of about 85%.Comment: 7 pages, 2 figures, 4 tables. Accepted at IEEE Symposium Series on
Computational Intelligence 202
Convolutional autoencoder-based multimodal one-class classification
One-class classification refers to approaches of learning using data from a
single class only. In this paper, we propose a deep learning one-class
classification method suitable for multimodal data, which relies on two
convolutional autoencoders jointly trained to reconstruct the positive input
data while obtaining the data representations in the latent space as compact as
possible. During inference, the distance of the latent representation of an
input to the origin can be used as an anomaly score. Experimental results using
a multimodal macroinvertebrate image classification dataset show that the
proposed multimodal method yields better results as compared to the unimodal
approach. Furthermore, study the effect of different input image sizes, and we
investigate how recently proposed feature diversity regularizers affect the
performance of our approach. We show that such regularizers improve
performance.Comment: 5 pages, 1 figure, 4 table